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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

• If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )S Responsive to communication(s) filed on 02/01/2000 . 
2a)D This action is FINAL. 2b)S This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) I3 Claim(s) 1-17 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 1-17 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 02/01/2000 is/are: a)E3 accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

1 3) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 1 9(a)-(d) or (f). 

a)DAII b)D Some*c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) |~] Notice of Drafts person's Patent Drawing Review (PTO-948) 5) CD Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 



1. 



This action is responsive to communications: application filed on 02/01/00. 



2. 



Claims 1-17 are pending in the case. Claims 1,8, and 10 are independent claims. 



Claim Rejections - 35 USC § 103 



3. The following is a quotation of 35 U.S. C 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

4. Claims 1-17 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Srivastava et aL, US 6,549,922 Bl fded 10/01/1999 in view of Sidana, US 6,571,295 Bl 
priority filed 01/31/1996. 

Regarding independent claim 1, Srivastava teaches a means for analyzing a web page 
to identify at least one markup tag to extract metadata from the multimedia file in Fig. 1 . 
Srivastava also teaches a means for selecting and executing a media processing program for 
analyzing the content of the multimedia data supplied by a resource to generate metadata 
describing its content in Fig. 1 and col. 2 lines 45-50. Srivastava teaches a means for formatting 
the metadata into a character-based text annotation in Fig. 5 col. 4 lines 8-13. Srivastava also 
teaches indexing which can be used for searching in col 8 lines 48-52. What Srivastava does not 
teach is a means for combining the web page with the annotation to form an enhanced web page. 

Sidana does teach a means for combining the web page with the annotation to form an 
augmented (same thing as enhanced) web page in Fig. 5 and col. 2 lines 13-30. It would have 
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been obvious to one of ordinary skill in the art at the time the invention was made to combine 
Sidana into Srivastava to create the claimed invention. It would have been desirable to combine 
both the web page and the annotations into one file which can then be conveniently viewed by 
the user. This would facilitate the user's increased understanding of the content contained in the 
web page. 

Regarding dependent claim 2, which is dependent on claim 1, Srivastava and Sidana 
teach the limitations of claim 1 as explained above. Srivastava teaches in col. 4 lines 32-48 a 
means for determining the particular data type of the multimedia supplied by the given resource 
and also a means for selecting a processing program for analyzing the multimedia data formatted 
in accordance with said particular data type. 

Regarding dependent claim 3, which is dependent on claim 1, Srivastava and Sidana 
teach the limitations of claim 1 as explained above. Srivastava teaches in the abstract a means 
for generating text data annotation expressed in accordance with the Extensible Markup 
Language. 

Regarding dependent claim 4, which is dependent on claim 1, Srivastava and Sidana 
teach the limitations of claim 1 as explained above. Srivastava teaches in Fig. 1 and col. 2 lines 
50-56 a means for acquiring metadata which describes the multimedia data supplied by a given 
resource. In Fig. 2 and col. 4 lines 8-13 Srivastava also teaches a means for including the 
additional metadata in a character-based text annotation. 

Regarding dependent claim 5, which is dependent on claim 4, Srivastava and Sidana 
teach the limitations of claim 4 as explained above. Srivastava teaches that the multimedia file 
displayed in Fig. 1 which will obviously be contained within a markup tag on an internet web 
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page and further in col. 2 lines 48-50 that some of the additional data includes information 
obtained from the markup tag. 

Regarding dependent claim 6, which is dependent on claim 4, Srivastava and Sidana 
teach the limitations of claim 4 as explained above. Srivastava teaches in Fig. 1 that a given 
resource may be accessed through the operating system of a computer and that some of the 
additional data may include information obtained from the operating system. 

Regarding dependent claim 7, which is dependent on claim 4, Srivastava and Sidana 
teach the limitations of claim 4 as explained above. Srivastava teaches in Fig. 1 that some of the 
additional information may be obtained via the internet. 

Regarding independent claim 8, Srivastava teaches a parser for identifying markup tags 
in web pages in col. 2 lines 46-48. Srivastava also teaches a processing means for analyzing the 
content of multimedia data identified by the mark up tags to generate metadata describing the 
multimedia data in Fig. 1 and col 2 lines 46-50. Srivastava teaches a means for translating the 
metadata into a character-based text annotation describing the multimedia data in Fig. 2 and col. 
4 lines 8-13. What Srivastava does not teach is a means for storing the combination of the web 
page and annotation to form an enhanced web page suitable for processing by text-based 
indexing and searching. Sidana does teach a means for storing the combination of the web page 
and annotation to form an enhanced web page suitable for processing by text-based indexing and 
searching in Fig. 5 and col. 2 lines 13-30. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Sidana into Srivastava to create the claimed invention. It would have been 
desirable to combine the annotation with the web page from which the multimedia file came to 
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create an augmented, or enhanced, web page which could be more easily understood by the user. 
This enhanced web page would decrease the amount of time required for the user to get the 
information from the web page which they need. 

Regarding dependent claim 9, which is dependent on claim 8, Srivastava and Sidana 
teach the limitations of claim 8 as explained above. Srivastava teaches in the abstract that the 
text annotation is expressed in the Extensible Markup Language. 

Regarding independent claim 10, Srivastava teaches in Fig. 1 and col. 2 lines 46-48 
identifying one or more mark up tags in a web page which respectively identify one or more 
external resources which provide multimedia data. Srivastava also teaches in Fig. 1 and col. 2 
lines 46-55 generating metadata which describes the multimedia data. Srivastava teaches in Fig. 
2 and col. 4 lines 8-13 translating metadata into a character-based text annotation. What 
Srivastava does not teach is inserting the annotation into the web page to form an enhanced web 
page suitable for processing by a character-based text processing system. Sidana does teach in 
Fig. 5 and col. 2 lines 13-30 inserting the annotation into the web page to form an enhanced web 
page suitable for processing by a character-based text processing system. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to combine Sidana into Srivastava to create the claimed invention. It would have been 
desirable to insert the annotation with the web page from which the multimedia file came to 
create an augmented, or enhanced, web page which could be more easily understood by the user. 
This enhanced web page would decrease the amount of time required for the user to get the 
information from the web page which they need. 



Application/Control Number: 09/496,086 Page 6 

Art Unit: 2176 

Regarding dependent claim 11, which is dependent on claim 10, Srivastava and Sidana 
teach the limitations of claim 10 as explained above. Srivastava teaches identifying markup tags 
in a web page in Fig. 1 . The multimedia file obviously is contained with markup tags on the v/eb 
page. Srivastava also teaches extracting the uniform resource locator of one of the external 
resources from at least one of the selected mark up tags in col. 3 lines 63-67. 

Regarding dependent claim 12, which is dependent on claim 10, Srivastava and Sidana 
teach the limitations of claim 10 as explained above. Srivastava teaches retrieving multimedia 
data from one or more external resources in Fig. 1 and analyzing the content of the multimedia 
data to extract metadata therefrom in col. 2 lines 46-55. 

Regarding dependent claim 13, which is dependent on claim 12, Srivastava and Sidana 
teach the limitations of claim 12 as explained above. Srivastava teaches identifying the data type 
of the multimedia data from each resource in col. 4 lines 32-35 and selecting a process routine 
for multimedia of the identified data type from each of said resources in col. 4 lines 39-48. 

Regarding dependent claim 14, which is dependent on claim 10, Srivastava and Sidana 
teach the limitations of claim 10 as explained above. Srivastava teaches indexing what may be 
an enhanced web page in light of claim 10 rejection to provide access to the web page in 
response to queries expressing one or more attributes expressed in the text annotation in col. 8 
lines 49-52. 

Regarding dependent claim 15, which is dependent on claim 10, Srivastava and Sidana 
teach the limitations of claim 10 as explained above. Srivastava teaches searching the content of 
what may be an enhanced web page in light of claim 10 rejection in response to a search request 
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to determine if attributes expressed in said search request are contained in the text annotation in 
col. 8 lines 49-52. 

Regarding dependent claim 16, which is dependent on claim 13, Srivastava and Sidana 
teach the limitations of claim 13 as explained above. Srivastava teaches indexing what may be 
an enhanced web page in light of claim 10 rejection to provide access to the web page in 
response to queries expressing one or more attributes expressed in the text annotation in col. 8 



Regarding dependent claim 17, which is dependent on claim 13, Srivastava and Sidana 
teach the limitations of claim 13 as explained above. Srivastava teaches searching the content of 
what may be an enhanced web page in light of claim 10 rejection in response to a search request 
to determine if attributes expressed in said search request are contained in the text annotation in 
col. 8 lines 49-52. 



The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. Burner et al, US 6,282,548 filed 06/21/1997 discloses a method and apparatus for 
simultaneously displaying a web page and metadata about the web page. Cheng et al., US 
6,519,597 Bl discloses a method and apparatus for indexing structured documents with rich 
datatypes. 



lines 49-52. 



Conclusion 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter J Smith whose telephone number is 703-305-593 1 . The 
examiner can normally be reached on Mondays-Fridays 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph H Feild can be reached on 703-305-9792. The fax phone numbers for the 
organization where this application or proceeding is assigned are 703-746-7239 for regular 
communications and 703-746-7238 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 703-305-3900. 



PJS 

June 20, 2003 




